Selection Bias in College Admissions Test Scores

Data from college admissions tests can provide a valuable measure of student achievement, but the non-representativeness of test-takers is an important concern. We examine selectivity bias in both state-level and school-level SAT and ACT averages. The degree of selectivity may differ importantly across and within schools, and across and within states. To identify within-state selectivity, we use a control function approach that conditions on scores from a representative test. Estimates indicate strong selectivity of test-takers in “ACT states,” where most collegebound students take the ACT, and much less selectivity in SAT states. To identify withinand between-school selectivity, we take advantage of a policy reform in Illinois that made taking the ACT a graduation requirement. Estimates based on this policy change indicate substantial positive selection into test participation both across and within schools. Despite this, schoollevel averages of observed scores are extremely highly correlated with average latent scores, as across-school variation in sample selectivity is small relative to the underlying signal. As a result, in most contexts the use of observed school mean test scores in place of latent means understates the degree of between-school variation in achievement but is otherwise unlikely to lead to misleading conclusions. * Corresponding author. Harris School of Public Policy Studies; 1155 E. 60 Street; Chicago, IL 60637. Tel: (773) 834-0207. Fax: (773) 702-0926. E-mail: [email protected]. We thank DeForest McDuff for excellent research assistance and Jeff Smith for helpful comments.